Modern color CCD and CMOS image sensors use a mosaic of color filter array, (CFA), material over a 2D array of photo detectors. This allows visible light centered on the wavelengths associated with visible light humans perceive as red, green, and blue to be measured. Instead of three separate N×M arrays for each of the colors, a single array with a mosaic pattern of CFA materials is used. Some positions in the array measure the green signal and others the red and blue. For each (x,y) position in the array, two of the three components needed for a 24-bit RGB spatial location are missing. The Bayer pattern and synthesized 24-bit color is illustrated in FIG. 15.
Color synthesis or “demosaicing” techniques exist at the heart of the RAW camera processing sequence. The techniques, if designed well, are robust in the presence of a wide variety of scene content, and if designed poorly, they are often considered the root cause of annoying and undesirable artifacts. The literature is full of techniques developed with varying levels of complexity over the past 10-15 years.
Color data synthesis techniques are designed to generate the missing red, green, and blue data components of a color image. This effectively allows three N×M arrays to be generated from a single N×M input array as shown in FIG. 15. The Bayer pattern CFA is one of the most popular formats for digital color imaging. A green filter material is applied to half of the array by interleaving it with red and blue material. For an array size of N×M, there are a total of N×M/2 green elements, and N×M/4 elements each for the red and blue channels. The synthesis of the missing color data components at each location in the array may be accomplished using both standard and proprietary techniques. In general, the techniques will display varying degrees of both complexity and visual quality. Depending on the intended purpose, both low and high complexity techniques may be used in an image capture system. The simplest techniques display a well-structured sequence of operations using a fixed access or filtering pattern for every spatial location. The more complex techniques often include an adaptation scheme based on some measured image parameter. Because artifacts tend to occur more often around edges, edge-based adaptation is fairly common and can yield drastic improvements in visual quality. However, using edge strength for adaptive control can require significant computational overhead.
Color data synthesis is the point in a RAW processing sequence where the raw Bayer sampled data is converted to the well-known RGB triads processed by other aspects of the image/video display and compression systems. It is the point in the processing where there is no return, and failure to deliver the optimum result can only be mitigated by downstream processing steps. Thus, it would be desirable, when synthesizing RGB color data from the raw Bayer CFA data to minimize edge artifacts and minimize aliasing artifacts. It would also be desirable to identify regions which are homogeneous and that contain structure which is related to “texture.”